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Abstract 

We propose a model based on SU(3) family symmetry which leads to a successful de- 
scription of quark and lepton masses and mixing angles including approximate bi-maximal 
mixing in the neutrino sector suitable for the LOW or quasi- vacuum solar solutions, with 
the atmospheric angle predicted to be accurately maximal due to the SU{3) symmetry. 
The model predicts a CHOOZ angle #13 ~ \ V u b\. The SU(3) symmetry can also ensure the 
near degeneracy of squarks and sleptons needed to avoid large flavour changing neutral 
currents. 
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1 Introduction 



The origin of fermion masses and mixing angles has been a long standing puzzle [0]. Our de- 
termination of the mass matrices has been improved by new experimental information on both 
the quark and lepton sectors. Particularly intriguing is the appearance of an almost maximal 
neutrino mixing angle #23 = 45° responsible for atmospheric neutrino oscillation and the sugges- 
tion of a near maximal mixing angle #12 ~ 45° in order to describe solar neutrino oscillation 0. 
Apart from the CHOOZ angle this contrasts sharply with the quark mixing angles which are all 
small. 

In this paper we shall show that a near maximal mixing angle may be a signal of an underlying 
non-Abelian family symmetry and demonstrate how vacuum alignment in such a model can lead 
to maximal mixing. Further we argue that the quark mass matrix is also indicative of a non- 
Abelian symmetry, although in this case the symmetry is responsible for the smallness of quark 
mixing angles. The origin of this apparent contradiction is the see-saw mechanism which alters 
the form of the neutrino mass matrix from that of the quarks and charged leptons. 

Non-Abelian family symmetries have been considered before@. In supersymmetric theories 
they offer an elegant solution to the flavour problem, ensuring that the squarks are nearly 
degenerate and thus suppressing flavour changing neutral currents (FCNC). Given that the 
third family of quarks and leptons is much heavier, and that the dominant contribution to 
FCNC comes from the light generations, early attempts to develop such theories considered the 
non-Abelian group SU(2) acting on the first two generations only[||. However these theories 
do not offer an easy explanation for the near maximal neutrino mixing in the neutrino sector 
involving the second and third generations. Such mixing requires a correlation between the 
(2, 2) and (2, 3) elements of the mass matrix. This strongly suggests an underlying non-Abelian 
symmetry involving the third generation too. For this reason we consider here a SU(3) family 
symmetry acting on all three generations with the view to addressing both these questions. A 
gauged SU(3) family symmetry which could provide a dynamical origin for the three observed 
families of quarks and leptons has a long history in the literature ||. However as far as we are 
aware previous SU(3) models have not been able to use the symmetry to relate mass matrix 
elements after spontaneous symmetry breaking. Here we show how this can be done via a novel 
mechanism for vacuum alignment. 

In this paper we shall consider a particular class of SU(3) model which gives a successful 
description of quark masses and mixing angles, and simultaneously gives approximately bi- 
maximal leptonic mixing with 623 fixed to be almost maximal by a combination of 577(3) and 
vacuum alignment. In both cases the origin of the mixing angles is controlled by the non-Abelian 
vacuum structure relating the second and third generations^. 

It is instructive to see how data favours such a structure in the quark mass matrix. A recent 
analysis of all experimental measurements relating to quark masses and mixings including the 
latest measurements at BaBar and BELLE has found the following form for the up and down 
quark mass matrices 

b'e 3 c'e 3 \ 
b'e 3 e^' e 2 a'e 2 (1) 
? ? 1 j 

1 With an Abelian U(l) family symmetry the mixing angle #23 cannot be enforced to be accurately maximal, 
but it may be large. 
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6e 3 e^ e 2 ae 2 f (2) 
m>b \ ? ? 1 

The equality of the magnitudes of the (1, 2) and (2, 1) elements leads to the successful Gatto, 
Sartori, Tonin (GST) relation|lO| for a (1, 1) texture zero. The (3, 1) and (3, 2) elements are only 
weakly constrained because measurement of the quark masses and the CKM matrix does not 
provide enough information to determine the full quark mass matrices. The parameters of the 
up quark mass matrix are given by e = 0.05, V ~ 1 while a! and d are very weakly constrained. 
The parameters of the down quark mass matrix are much better determined with 

e = 0.15 ±0.01, b =1.5 ±0.1, a = 1.31 ± 0.14, 

0.4 ± 0.02, ip = 24° ± 3° or (3) 
1.27 ±0.05, ^ = -58° ±5° (4) 



|c| 
|c| 



where c = |c| e 1 ^. The fact that the (2, 2) and (2, 3) matrix elements are very similar in magnitude 
is required by the smallness of It suggests a relation between these elements, suggestive of 
a non-Abelian symmetry. The recent data now also requires that the (1, 2) and (1, 3) are also 
quite similar, supporting this interpretation and disfavouring the promising symmetric texture 
zero structure with zeroes in the (1,1) and (1,3) elements || |J0. 

In the present paper we shall propose a model of fermion masses and mixing angles based 
on an SU(3) family symmetry, together with certain discrete symmetries which are required to 
forbid unwanted operators. The model we propose will give rise to Yukawa matrices of the formQ 

/ 0(e 8 ) Ae 3 Ae 3 \ 
y~ -Ae 3 e 2 e 2 (5) 
V -Ae 3 e 2 1 J 

in leading order of the expansion parameter, e. The coupling A is not determined by the symme- 
try but is expected to be of 0(1). The matrix has a hybrid symmetry, being symmetric in the 
lower block (3,2) = (2,3) and antisymmetric in the remaining entries (2,1)=-(1,2), (3,1)=-(1,3). 
This form applies to the up and down quarks, to the charged leptons and to he neutrino Dirac 
couplings between left- and right- handed neutrino components. In general there is an indepen- 
dent expansion parameter e for each Yukawa matrix. Note that at leading order each Yukawa 
is described by only two free parameters! The right- handed Majorana coupling matrix has a 
different structure. 



2 The SU(3) family symmetry. 

Here we outline the structure of the model, postponing details to later sections. We start with 
an SU(3) family symmetry which commutes with the Standard Model (SM) gauge group. We 

2 If one allows for an asymmetric form of the quark mass matrices with large entries below the diagonal it is 
still possible to have a (1, 3) zero[0 . Here we concentrate on symmetric structures which naturally accommodate 
the successful GST relation. 

3 Such matrices have been considered before, eg sec Q, but not with a symmetry ensuring equality of matrix 
elements. 
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shall construct a supersymmetric version of the theory so, apart from the modifications needed 
to implement the family symmetry, we have the basic structure of the minimal supersymmetric 
SM (MSSM) in which fermions belong to chiral supermultiplets. For simplicity of presentation, 
we will treat the supersymmetric structure as implicit. The family symmetry assignments for 
the left-handed quarks and leptons are: 

1>i e (Qi, U) ~ 3, e (t?, A c , El N°) ~ 3 (6) 

where z = 1, 2, 3 are SU(3) labels. To build a viable model we need spontaneous breaking 

3U{3) — ► SU(2) — > Nothing (7) 

Note that whenever we write SU (3) it will always refer to the new family symmetry and 
SU (2) will always refer to its subgroup. These should not be confused with the SM gauge group 
factors with which the family symmetry commutes. 

The Higgs fields responsible for the above symmetry breaking are the £77(3) antitriplets (but 
SM singlets) 3 , 23 , which develop vacuum expectation values (VEVs) 



< 03 >= 




Note that < 03 > can always be rotated into the third position using SU(3). The alignment of 
the < 023 > VEVs is non-trivial and is the subject of a later section. 

The leading contributions to the Yukawa matrices arise from operators which are quadratic 
in 3 and 23 

^#23^23 + H a (9) 

where H a are the two Higgs doublets of the MSSM, which are £77(3) singlets, but carry the 
usual electroweak quantum numbers. If ip c = U c , N c then a = 2, while if ip c = D c , E c then 
a = 1,2. The difference between the mass scales M and M 3 is discussed below. 



2.1 The operator mass scale 

The origin of these operators lies in the physics beyond the SM at some higher energy scale and 
the inverse mass scale associated with these operators reflects this higher scale. For example 
the operator may be due to a Higgs "messenger" sector through mixing of the MSSM Higgs 



with heavy vectorlike Higgs fields |TT|] . In this case the inverse mass scale is that of the heavy 
Higgs. Since the heavy Higgs carrying the weak hypercharge of the H a in the a = 1,2 sectors 
may be different, the case when ip c = U c , N c is expected to have a different inverse mass scale, 
M = M u , from the case when ip c = D c , E c for which M = M d . The operators may also arise 
through mixing of the quarks with heavy vectorlike quarks but again we expect the associated 
operator mass scale to differ for the up quark and down quark sectors. When 2 3 is replaced 
by its VEV, the operators of eq@ generate equal (2,2) = (2,3) = (3,2) elements of 0(b 2 /M 2 ) in 
the 23 block of Eq.[5[ As we shall see later this equality is responsible for the maximal neutrino 
mixing angle 823- Due to the different inverse mass scale the expansion parameters in the up 
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and the down sectors are expected to be different. We shall denote the expansion parameter in 
the up sector by e = b/M u and that in the down sector by e = b/M d . 

Some comment about the fact that the leading term in Eq.[5] for both the up and the down 
sectors is of 0(1) is in order. At first sight it would seem that, because the (3,3) term comes 
from a higher dimension operator, its magnitude should differ in the up and down sectors for 
the same reason that the 0(e 2 ,e 2 ) terms differ. However if a 3 > M, where M is the generic 
mass scale associated with the up or the down sector, the expansion parameter is not a 3 /M 

but rather a 3 /M 3 where M 3 m \J M 2 + a|. The reason for this is that the heavy messenger 
field responsible for generating the higher dimension operator necessarily has a contribution 
to its mass coming from a 3 since it couples to 3 . In addition it will have other contributions 
characterised by M. Obviously the largest term dominates giving the result above. Here we 
assume that a 3 > M > b so that <p 3 provides the first stage of breaking of 577(3), while the 
second stage, triggered by the VEV of 02, is below the mass scale of the heavy sector and so 
generates small effects characterised by the expansion parameters, e and e. In what follows we 
shall often refer to the expansion parameter as e but it should be remembered that this will be 
e in the down quark sector. In this paper we shall not consider the details of the heavy sector 
responsible for the generation of the operators but content ourselves with the construction of the 
effective low energy theory, including operators consistent with the symmetries of the theory. 



2.2 Subleading operators 

The operators which are mixed in </> 3 , 23 give the order e 3 contributions to the (2, 3), (3, 2), (3, 3) 
elements in Eq.|5], and so we require them to be suppressed by e 2 relative to the quadratic 
operators in Eq.^ 

" + r,o 3 r^ :1 )//,, (10) 



e 2 



MM- 
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If the mixed operators were not suppressed then they would imply that the (2,3) element is 
of 0(e), larger than the (2,2) element of 0(e 2 ),and would lead to the bad relation \V c b\ — 
^m s /mb. Here we require a further 0(e 2 ) suppression for reasons that will become apparent. In 
order to achieve this suppression we introduce a discrete symmetry Z 2 under which 3 and 23 
have opposite parity. This allows the quadratic operators, but forbids the leading order mixed 
operators. The latter are only allowed at higher order with the e 2 suppression factor. 

In order to generate Yukawa entries in the first row and column, we need to introduce the 
577(3) triplet (SM singlet) Higgs 3i , 23 i( which we shall show develops VEVs along D— and 
F— flat directions. 

<d) 3 >= | | , <0 23 >= I 6|. (11^ 
The operators responsible for the first row and column of the Yukawa matrices are then 

<2 (^i/HhajWH* (12) 




e 6 



M 2 M| 



M 
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The inclusion of triplet Higgs in addition to the antitriplets introduced previously allows the 
antisymmetric tensor e y ' fc to be used to generate the first row and column of Eq.[5|. The operator 
in Eq.|12| generates (1,2) = (1,3)=-(2,1)=-(3,1) entries at order e 3 , while the operator in Eq.|T3| 



generates the (1,1) entry at order e 8 . In the next section we discuss the origin of the required 
operator suppression factors. 

3 An SU(3) Model 
3.1 Operator Analysis 

In order to give a specific realisation of the basic scenario of the previous section we consider the 
full (SM singlet) scalar sector of the model summarised in Table |l|. To ensure that the operators 
are only allowed at the specified orders we have introduced an R-symmetry under which all 
superpotential terms are required to have R = 2. With these fields and VEVs introduced in 
the previous Section, the leading terms generating the fermion masses allowed by the family 
symmetries come from the superpotential 

p ~ (14) 

(Atts^l + ^^S^Mt^M^) (15) 









w 


+ 


1 


MfM 3 


+ 


1 


M|M 3 


+ 


1 


MfM 8 



MfM 



<Y (16) 
V^^)(e' mn « m ^J^a(^ 2 3, P ) 6 (17) 
l —- (e^^03,^ 3 S) 4 + H a (18) 



where the terms in Eqs. p^ - |T7| correspond to the operators in Eqs .|9|, |T0| , p~2| , p~3| . They generate the 
correct suppression factors once the fields are replaced by their VEVs appropriately. Eq . fll8|) 
includes a further subleading term that will be important in the following discussion. 

It is straightforward to show that the operators in P are the leading ones allowed by the 
symmetry. In Table |2| we list the operators of leading dimension which would spoil the form 
of eq(^]), together with the symmetry that forbids it. As we shall see in the next section, we 
may take the 2 VEV to be very small and so operators involving this field may be ignored. The 
remaining operators are forbidden by the family symmetries. 



3.2 Vacuum Alignment 

The critical feature of the model is the vacuum alignment which arranges for the fields 023, 023 
to acquire VEVs of equal magnitude in the 2 and 3 directions. Here we discuss how this comes 
about. The initial stage of symmetry breaking is triggered by VEVs for 3 and 2 - We assume 
that the fields have Yukawa couplings to the heavy sector fields which drive soft mass squared 
terms negative at some scale, A, through radiative corrections. The VEV cannot be larger than 
this scale as the effective potential has the form m 2 (<j) 2 )<j) 2 and clearly is positive above this scale 



6 



Field 


SU(3) 


z 2 


R 


L 




3 


+ 


+ 1 





dn 

TO 


3 


+ 


-2 


o 


Y2 


3 


1 


o 


o 


Y23 


3 




+ 1 


o 


H J 26 


3 


1 


o 


o 


V 


3 


+ 


+ 1 


1 


V 


3 


+ 


+ 1 


-1 


u 


1 




1 





X 


1 


+ 


+ 1 





Y 


1 


+ 


+2 





z 


1 







1 



Table 1: Transformation of the Higgs superfields under the family symmetries. 



Category 


Forbidden Operator 


Reason 


I 


^023^03^ 
^023^03#a(03Z0 3 ) 
^023^03#a(023*0 3 ) 
^023^i03#a(03«0 23 ) 


z 2 

Z 2 ,R 
Z 2 ,R 
R 


II 


e ijk *l>i(j)23ji> C kH a 
e ijh ^i(j)23MH a ((f) 3 i(f) l 3 ) 
e ijk ^i<j)23MH a ((f) 23l <j) l 3 ) 
e ijk i>i<p 23j i) c k H a (<p 3l 4> 2Z ) 


R 
R 
R 

Z 2 ,R 


III 


e ijk ^ i( j) 3j ilj c k H a 
e ijk ^ i( j) 3j ilj c k H a ((f) 3l (f) l 23 ) 


R 
R 
R 

Z 2 ,R 


IV 


(e ijfc ^023,03fc)(^ C 03)^ 
(6^023,03,0 (#023 ) (M%)Ha 
(e lJ Vi023j03fe)(^f023)^ 


R 
R 

Z 2 ,R 


V 


(e ljfe ^023 J 3fe )(e' mri ^ c 023 m 03„)^ 


R 



Table 2: Forbidden operators and the reason why they are excluded. 
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and negative below. We suppose that such radiative effects trigger VEVs for 3 and 2 . Without 
loss of generality we can choose the basis such that 0g = f 03 ) . The alignment of the 

VEVs of 03 and 02 is due to the term in the superpotential 

P x ~ X^f 2 (19) 

such that along the F-flat direction \Fx\ = 0, 03 and 02 are orthogonal. Again without loss 
of generality we can choose the basis such that 02 = ( a 2 J . As we demonstrate in the 

Appendix, for a particular range of parameters, D— flatness requires 3 = ( 03 ) . 

Consider now the field 2 3- We assume that due to different heavy sector interactions, its 
mass squared remains positive and its VEV is triggered by an F-term. The SU(3) symmetry 
guarantees the equality of the components of the soft mass term mf 3 |023| 2 = m 23(l023,i| 2 + 
1 023,2 1 2 + 1023.3 1 2 )- We will use this to obtain the vacuum alignment required. Consider the 
superpotential terms consistent with the symmetries of Table [T]. 

P 2 ~ V(0 23 0202303 ~ ^) , ^3 ~ £7023023 (20) 

Here \i is a mass scale associated with spontaneous symmetry breaking in the heavy sector. 
Requiring \Fy\ = implies < 023,2023,3 >= Ai 4 /(°2 a 3) = b 2 for > 0f\ This is the vacuum 
alignment that will lead to maximal mixing in the neutrino sector. The resultant pattern of 
symmetry breaking along 7n| 3 D— and F— flat directions, for the range of parameters detailed 
in the Appendix, is given in Table || Note that the fact the magnitude of the two VEVs in 2 3, 
023 are equal follows from the minimisation of m| 3 |023| 2 and that it is the underlying SU(3) 
which ensures this by ensuring the soft mass terms are equal for all three components. 

Finally we consider the fields Q is, V responsible for the Majorana masses. Below the scale 
of 577(3) breaking, a 3 , only the SU(2) gauge bosons are light and contribute to the running 
of the v masses. Thus, below this scale, V\ 2 become heavier than T> 3 . However is may also 
have Yukawa couplings to heavy states, which need not feel large SU(3) breaking if they do 
not couple to 03 in the superpotential. Such terms will drive ml negative in the usual way, 
driving a VEV in the lightest z? 3 . Thus it is easy to align the VEV of v without the addition 
of any superpotential "alignment" terms. However, if such terms are present they will skew the 
VEV from the direction favoured by the soft terms because such an F-term is not suppressed 
by the SUSY breaking scale that characterises the soft mass terms. When we come to consider 
neutrino masses it will be necessary to generate a component of the VEV along the v\ direction. 
It is easy to construct just such a term via the introduction of an F— term consistent with the 
symmetries. Suppose we have the field Z with quantum numbers shown in Table || The allowed 
superpotential terms are 

P 4 ~ Zz/( e ^30 3 fe (0303) + 023;(0302 3 ) 2 (03023)) (21) 

Now the vanishing of Fz implies jz/ 1 — e^z/ 3 >= so the VEV is skewed in the direction \v >= 

(e 2 ,0,lK 

4 We take b to be real, although our analysis is not sensitive to this assumption. 

5 These lepton number violating Higgs superfields should not be confused with neutrino superficlds. 
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Table 3: Vacuum expectation values. Phases are not shown 
3.3 Quark masses 



The operators of eqs(14-17) with the VEVs of Table |] generate the Yukawa matrices of the 
form given in eq(^). This is of the correct form to reproduce the "experimental" form for the 
mass matrices given in eqs(|l]) and once the appropriate expansion parameters are inserted 
as discssed in Section 271. Note that the leading operators require|6'| = |c'| , |6| = |c| , \a'\ = 1, 
\a\ = 1. This follows from the vacuum alignment of the 023, 2 3 fields. However sub leading 
operators will spoil these relations. Keeping all the contributions in eqs(|TJj-[I3|) we obtain the 
form 



M 
m 3 



Ae 3 



-Ae 3 + 0(e 4 ) 
-Ae 3 + 0(e 4 ) 



e 2 + A'e 



'^3 




(22) 



For the case of the down quark mass matrix the expansion parameter is not very small, e ~ 0.15 
and so the corrections can be quite large. Using this freedom we see that the SU(3) model is 
able to reproduce eqs(|l]) and (0) and thus the quark masses and mixing angles. 



3.4 Charged lepton masses 

With the lepton assignments to SU(3) triplets as in eq(|6]), and assuming the same Z 2 and R 
charges as for the quarks, the lepton mass matrix will have the same form as the of the quarks, 
although the expansion parameter may differ if the mass scale associated with the operators is 
different. However, since the charged leptons get their mass from the same Higgs as generates 
the down quark masses, if the operators are dominantly generated by mixing to Heavy Higgs 
fields the expansion parameter for down quarks and charged leptons will be the same. The same 
will be true if the heavy charged leptons have the same mass scale as the heavy down quarks. 
Here we assume this is the case and so one should use the expansion parameter e in eq (|22|) 
when computing the lepton masses. However it is not consistent to have identical down and 
charged lepton mass matrices. The relation = m T is consistent with the measured values 
after radiative corrections are included, provided this equality applies at a very high scale (the 
GUT scale?). However to describe the lighter generations it is necessary to have the approximate 
relations m s ~ and ~ 3m e at the high scale. This can be achieved through the choice 

of different Yukawa couplings for the charged leptons and down quarks and it may be that this 



choice follows from an underlying GUT as suggested by Georgi and Jarlskog |TI|. Since we are 
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not attempting here to construct the underlying theory valid at a high (GUT) scale, we will not 
pursue this possibility further but merely note that the choice of lepton Yukawa matrix of the 
form 

/ e 8 Ae 3 Ae 3 \ 
Y { =\ -Ae 3 3e 2 3e 2 (23) 
V -Ae 3 3e 2 1 / 

provides an excellent fit to the lepton masses. The form is that required by the underlying 
SU(3) symmetry and we have included a factor 3 to achieve the desired relations for the two 
light generations while keeping the other couplings the same as for the down quarks. 



3.5 Neutrino masses 

We turn now to neutrino masses. As we shall discuss the choice of vacuum alignment given 
above allows for near bi-maximal mixing even with the universal form for the Dirac masses! 
Due to the possibility that neutrinos have Majorana masses, the form of the light neutrino mass 
matrix is expected to differ substantially from that of the quarks and charged leptons. The 
general form of the effective mass matrix is given by the see-saw form 

m eff = m LR .MxR.ml R (24) 

where is the 3x3 matrix of Majorana masses for the three generations of right-handed 

neutrinos and mm = Y u v 2 , where v 2 is the VEV of the second Higgs doublet, and Y u is the 
neutrino Yukawa matrix coupling the left- to right- handed neutrino components. Here we 
assume there are no Iw = 1 Higgs fields giving rise directly to a Majorana mass for the left- 
handed neutrinos. 



3.5.1 The Dirac mass matrix 

Given the neutrino £77(3) assignments of eq@ we see that the Dirac mass matrix must be of 
the same form, eq(|22|) as the charged lepton and quarks. However the expansion parameter may 
of course be different. Since in this case the neutrinos get their Dirac mass from the same Higgs 
as generates the up quark masses, if the operators are dominantly generated by mixing to Heavy 
Higgs fields the expansion parameter for up quarks and neutrinos will be the same. The same 
will be true if the heavy neutrinos have the same mass scale as the heavy up quarks. We assume 
this is the case here. As a result the Yukawa matrix is given by 

/ e 8 A,e 3 + A^e 4 A,e 3 + A'J'e 4 \ 

Y v = -A,e 3 + A^e 4 a„e 2 a u e 2 + A'.e 3 (25) 
V -A,e 3 + A^'e 4 a u e 2 + A'.e 3 1 J 

where we have included the couplings of 0(1). 

3.5.2 The Majorana mass matrix 

The dominant (heavy right-handed) Majorana mass term comes from 

p 5 ~ ^l>yvH] (26) 
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which gives Majorana mass, M 3 = a 2 /M, to the third family and mass Mi = ef,a 2 /M, to the 
first family. There are further Majorana masses generated higher dimension operators allowed 
by the symmetries. The leading ones are 



M,M 5 



?23,Z 



(27) 



leading to the final form for the (heavy right-handed) Majorana mass matrix given by 



M, 



RR 



M, 



RR,33 









■2 



The result of this is to give (heavy right-handed) Majorana masses, M\ : M 2 : M 3 = e 4 : e 3 : 
1 where we have allowed for a new expansion parameter e v . Note that if e v << e the mixing 
in the light neutrino sector will be dominated by the Dirac mass matrix. In this case the three 
Majorana mass eigenstates are along the 1, 2 and 3 directions in the basis in which Mp is 
written. This alignment proves crucial in allowing for near bi-maximal mixing. 



3.5.3 The effective mass matrix for the light neutrinos 

The see-saw mechanism gives a light effective Majorana matrix via the see-saw formula of eq(p4| 
The resulting Lagrangian giving the light doublet neutrino masses is then 

r (eV + (-A.6 3 + \y) + (-A,e 3 + A^'e 4 ) v T f 2 

M x V 
| ({a u e 2 + A^e 3 ) v T + a v e 2 v^ + A^e 3 ^) 2 ^ 



M 2 

[v T + (a u e 2 + A^e 3 ) + \ u e 3 v e ) 2 
M, 



^ yu T -r yu, v Ki -r s\ v <c j is ^ ~r s\yt u e) ^ 



Maximal atmospheric neutrino mixing follows if we have single right-handed neutrino dominance 
]T3| by choosing the parameter e u such that the first right-handed neutrino gives the dominant 
contribution to the 23 block of the light effective Majorana matrix. The condition for this is 
e v < e 2 < 1 and is sufficient to guarantee that the contributions from the second term along 
the 2 direction in eq(^) is the next most important and determines the mass and mixing of the 
second heaviest neutrino state. The resulting mass matrix has heaviest state v a = {y^ + v T )/y/2 
with mass oc e 6 /e 4 . Up to the correction terms of order e this is maximally mixed (45°) due to 
the vacuum alignment of 023- 

Turning to the lighter neutrinos the second term in eq(|29|) provides the dominant contribution 
if e u < e 2 . A novel feature of the SU(3) structure is that it leads to the possibility of near maximal 
mixing in solar neutrino oscillation too! Due to the SU (3) symmetry the leading contribution 
from this term is a 2 e 4 (z/ M + u T ) 2 /M 2 and thus just adds to the v a mass. At subleading order it 
generates a mass m 2 oc e 6 /e 3 to an orthogonal component to v a which is a mixture of — v T 
and v e at the same order in e. As a result there will be large mixing in this sector too. The ratio 
of neutrino masses is given by 

^~e,<e 2 (30) 
m 3 
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With this ratio our model may be consistent with either the LOW or quasi-vacuum oscillation 
solutions but not with the LMA solution. With only two right-handed neutrinos effectively 
contributing, with the first one being dominant, and the second being subdominant, we may use 



the analytic results of [14] to estimate the remaining mixing angles in this model, which are all 
expressed in terms of neutrino Yukawa matrix elements in Eqf25| . We have already seem that 
the atmospheric mixing angle #13 is given by 

tan# 23 = ^i = l (31) 

with small corrections of order \V c b\ from the charged lepton sector. The contribution to the 
CHOOZ angle from the neutrino sector 9^ is predicted to be vanishingly small 

*?3 = 1 = I ~ ^ (32) 

Therefore the CHOOZ angle will originate from the charged lepton sector and from eq.(p3|) we 
predict a CHOOZ angle 

#13 ~ \V ub \ ~ (e) 3 ~3x 1(T 3 . (33) 

The solar angle #12 is predicted to originate mainly from the neutrino sector, with small 
corrections of order the Cabibbo angle from the charged lepton sector. Neglecting the small 
CHOOZ angle, and inserting the maximal atmospheric angle, we find 

\/2Y u 

tanfo=-£— ^--0(1) (34) 

r 22 r 23 

where the leading contribution to Y 2 2 — ^23 cancels due to vacuum alignment, and the subleading 
e 3 term is of the same order as Y" 2 but comes from a different operator with an independent 
coefficient, so the solar angle is not precisely predicted but is expected to be large. 

Note that the large solar mixing angle is due to the equality, in leading order, of the (2, 2) 
and (2, 3) matrix elements of Y u , which followed from the vacuum alignment made possible by 
the underlying SU (3). This in turn was motivated by the structure of the quark mass matrices. 
In particular the near equality of the (2, 2) and (2, 3) Yukawa matrix elements followed from 
the smallness of V^. The possibility that the (2, 2) and (2, 3) matrix elements should be equal 
inleading order has been suggested by the recent data, particularly on the B s lifetime 0. 



3.6 Soft Mass Terms 

One of the main motivations for a non-Abelian family symmetry is the need to solve the flavour 
problem in supersymmetric models, suppressing large contributions to FCNC coming from vir- 
tual diagrams involving non-degenerate squark and slepton masses. Here we consider whether 
the SU(3) family symmetry is able to achieve this. Generically soft mass terms arise from D- 
terms of the form (ipfyiS^ S) d where S is some supersymmetry breaking singlet field which has 
a non- vanishing F-term F$, leading to soft scalar masses \Fg \ 2 (tpj^i)/M' 2 . The SU(3) family 
symmetry therefore ensures that the leading order soft scalar masses are proportional to the unit 
matrix in family space. However the SU(3) breaking Higgs fields will lead to important correc- 
tions to the soft masses. The leading correction is obtained from ((i^i4>\){'^j4 > iy S) d / M lA . This 
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leads to a contribution to the third family soft scalar masses suppressed by a factor 0(a 3 /M'). 
The second family receives soft scalar mass corrections from ((^^^('ipjifi^y S)d/ M' A corre- 
sponding to a suppression factor of 0(b/M'). Here we have allowed for a new mass scale, M', 
associated with these operators. 

The detailed dynamical origin of the fermion mass operators discussed above may play a role 
in determining the magnitude of M'. For example if the fermion mass operators originate from 
a Higgs messenger sector, then the higher order corrections to the leading soft operators could 
be highly suppressed, M 1 » M because they do not involve the Higgs sector. If they originate 
from fermion messenger sectors then, since the expansion parameter is larger in the down sector 
than in the up sector, this may lead to right-handed scalar down mass corrections dominating 
over right-handed scalar up mass corrections, and so on. Provided M' > M u , the FCNC should 
be adequately suppressed due to the underlying SU (3) symmetry. 

The SU(3) breaking effects can also give important corrections to vacuum alignment. Recall 
that the equality of the soft mass components ^23,2^23,2 = m 23,3023,3 was crucial in obtaining 
the vacuum 23 = (0, b, b). The operator 

^((023^X023^^ (35) 

can give large corrections to the third component soft mass ^23,3 spoiling the vacuum alignment 
mechanism. Allowing for this splitting one finds 2 3 = (0,62,^3) where 62/^3 = m 23 2/ m 23 3- ^ n 
order for the vacuum alignment to be preserved it is necessary for a 3 /M" to be small. Whether 
this is the case depends on the details of the heavy messenger sector. 

In summary SU (3) offers an elegant mechanism to make the soft quark and lepton mass terms 
sufficiently close in mass to avoid unacceptably large contributions to FCNC. Whether the SU(3) 
protection mechanism is sufficiently robust depends on the details of the heavy messenger sector. 



4 Summary and Conclusions 

In this paper we have constructed a model of fermion masses based on an SU (3) family symmetry. 
The novel feature of this implementation is the use of the underlying SU(3) to align the vacuum 
expectation values of the fields spontaneously breaking SU(3). This provides a mechanism for 
explaining maximal mixing in the atmospheric neutrino sector in the case of just three light 
neutrinos. Our model predicts either the LOW or the quasi-vacuum solar solutions and a 
CHOOZ angle of order V u b. Allowing only for unknown Yukawa couplings of 0(1), we were able 
simultaneously to fit the quark and lepton masses and mixing angles in a model in which all 
left-handed and charge conjugate right- handed fermion states belonged to 577(3) triplets. Given 
that the lepton mixing angles are large while the quark mixing angles are small it is remarkable 
it is possible to achieve such a degree of similarity between quarks and leptons. The reason is 
due to the structure of the neutrino mass matrix and the see-saw mechanism which provides 
the mechanism to obtain large lepton mixing angles from the structure responsible for small 
quark mixing angles. A bonus of the scheme is the protection the 577(3) family symmetry offers 
against large FCNC. 

The model we have constructed is the low energy effective theory coming from Beyond the 
Standard Model physics. While it is of interest to constuct the underlying theory, it is clear that 
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there is considerable flexibility in its structure. For example we can construct a Higgs messenger 
model in which all the higher dimension operators responsible for the light fermion masses comes 
from mixing with heavy Higgs states. Alternatively there are examples in which an underlying 
string theory gives rise to the same effective low energy structure discussed here. We hope to 
address these questions in a future publication. 



Appendix - Vacuum Alignment 



Following Section 3.2 we start with the vacuum structure 









( 


) 


< 03 > = 




, < 02 > = 












\ 


o J 



(36) 



which is to be triggered by radiative corrections. We assume that m\ becomes negative at a 
much larger scale than m\ so that ultimately we will have a\^> a\. For radiativre corrections 
to trigger a large symmetry breaking scale in a supersymmetric theory the VEVs must develop 
along D-flat directions. Suppose there are scalar fields transforming as 3 under £77(3) and that 
03 has the smallest soft mass amongst these. Then if m|(A 2 ) +m|(A 2 ) < at some scale A then 



a VEV for 



will appear and to leading order D-flatness will align it with < 



>, 



< 




(37) 



where a 3 ~ A. 

Consider now the additional fields 2 3, 023- Throughout the analysis the important effect D- 
terms must be taken into account, since these terms play an important role in determining which 
VEVs occurs and in vacuum alignment. We assume that 023 has a positive mass squared and its 
VEV is triggered by minimising Fy from Eq.EO which leads to < 023,2023,3 >= /-t 4 /( a 2 a 3) = b 2 . 



The mass term rn| 3 <; 
and 3 directions, 



'23 



m. 



23 



(0; 



23,1 + 



7 23,2 



+ 



^3,3) 



< 



'23 



>= 



is then minimised by equal VEVs in the 2 



(3* 




Through the choice of the \x parameter in Eq.^0] we can arrange a hierarchy a 3 3> b ^> a 2 . Then 
a 2 can be made small enough so as not to affect the mass matrices significantly. 



Assuming mi < 



ni 



23 



minimising Fjj from Eql 



< 0, radiative symmetry breaking triggers a VEV for 
D] aligns it to be orthogonal to < 2 3 >, 

/ 



< 



'23 



< 023 > = 



> and 



(39) 



V 
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In order to determine b' we must allow for subleading corrections to the D-flatness conditions 
which align < 3 > with < 3 > f\ 




< 03 >= y • (40) 



Note that the Fy correction to </> 23 VEV is proportional to a 2 and hence negligible. 

The D-terms coming from the generators T 3 = diag(l, — 1, 0), T 8 = diag(l, 0, — 1), T 23 = 
S i2 S j3 , are: 

\D 3 f = |_ 2/ 2 + 6 2_ a 2_ 6 /2|2 = (41) 

\D 8 \ 2 = \ a j + b 2 -(a 3 + x) 2 -b' 2 \ 2 = (42) 
|D 23 | 2 = \b 2 + b' 2 -y(a 3 + x)\ 2 = (43) 

Solving for these we find, to leading order in 6/a 3 , that x — 0, y — 26 2 /a 3 , 6' = b, which leads 
to the VEVs in Table g. 

By a similar analysis one can check that the preferred vacuum has zero VEVs in the first 
components. Allowing for < 31 >= z, < 4> 2 3,i >= ct, < ^23,1 >= and considering the 
same D-terms as previously and in addition those associated with the generators T i2 = 5nSj2, 
T13 = Sn5js, and also considering the soft mass terms associated with the scalars 3 , 2 3, </>23, it 
can be shown that for m| 3 > 2m|g that a = a = z = 0, to leading order. 
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